In any complex information system that accepts unstructured or semi-structured input (such as an intelligence system) for the external work, it is obvious that change is the norm, not the exception. Media and data streams are often modified and otherwise constantly change making it difficult to monitor them. Moreover, in any system involving multiple users with divergent requirements, even the data models and requirements of the system itself will be subject to continuous and pervasive change. By some estimates, more than 90% of the cost and time spent on software is devoted to maintenance and upgrade of the installed system to handle the inevitability of change. Even our most advanced techniques for software design and implementation fail miserably as the system is scaled or is otherwise changed. The reasons for this failure arise, at least in part, from the very nature of accepted software development practice/process.
Referring now to FIG. 1, the root of the problem with the current software development process, which we shall call the “Software Bermuda Triangle” effect, is shown. Conventional programming wisdom holds that during the design phase of an information processing application, programming teams should be split into three basic groups. The first group is labeled DBA (for Database Administrator) 105. These individuals 105 are experts in database design, optimization, and administration. This group 105 is tasked with defining the database tables, indexes, structures, and querying interfaces based initially on requirements, and later, on requests primarily from the applications group. These individuals 105 are highly trained in database techniques and tend naturally to pull the design in this direction, as illustrated by the small outward pointing arrow. The second group is the Graphical User Interface (GUI) group 110. The GUI group 110 is tasked with implementing a user interface to the system that operates according the customer's expectations and wishes and yet complies exactly with the structure of the underlying data (provided by the DBA group 105) and the application(s) behavior (as provided by the Apps group 115). The GUI group 110 will have a natural tendency to pull the design in the direction of richer and more elaborate user interfaces. Finally the applications group 115 is tasked with implementing the actual functionality required of the system by interfacing with both the DBA and the GUI and related Applications Programming Interfaces (APIs). This group 115, like the others 105,110 tends to pull things in the direction or more elaborate system specific logic. Each of these groups tends to have no more than a passing understanding of the issues and needs of the other groups. Thus during the initial design phase, assuming a strong project and software management process rigidly enforces design procedures, a relatively stable triangle is formed where the strong connections 120, 125, 130 enforced between each group by management are able to overcome the outward pull of each member of the triangle. Assuming a stable and unchanging set of requirements, such a process stands a good chance of delivering a system to the customer on time.
The problem, however, is that while correct operation has been achieved by each of the three groups 110, 105, 115 in the original development team, significant amounts of undocumented application, GUI, and Database specific knowledge has likely been embedded into all three of the major software components. In other words, this process often produces a volatile system comprised of these subtle and largely undocumented relationships just waiting to be triggered. After delivery (the bulk of the software life cycle), in the face of the inevitable changes forced on the system by the passage of time, the modified system begins to break down to yield a new “triangle” 150. Unfortunately, in many cases, the original team that built the system has disbanded and knowledge of the hidden dependencies is gone. Furthermore, system management is now in a monitoring mode only meaning that instead of having a rigid framework, each component of the system is now more likely to “drift”. This drift is graphically represented by the dotted lines 155, 160, 165. During maintenance and upgrade phases, each change hits primarily one or two of the three groups. Time pressures, and the new development environment, mean that the individual tasked with the change (probably not an original team member) tends to be unaware of the constraints and naturally pulls outward in his particular direction. The binding forces have now become much weaker and more elastic while the forces pulling outwards have become much stronger. A steady supply of such changes impacting this system could well eventually break it apart. In such a scenario, the system will grind to a halt or become unworkable or un-modifiable. The customer must either continue to pay progressively more and more outrageous maintenance costs (swamping the original development costs), or must start again from scratch with a new system and repeat the cycle. The latter approach is often much cheaper than the former. This effect is central to why software systems are so expensive. Since change of all kinds is particularly pervasive in an intelligence system, any architecture for such systems would preferably address a way to eliminate this “Bermuda Triangle” effect.
Since application specific logic and it's implementation cannot be eliminated, what is needed is a system and environment in which the ‘data’ within the system can be defined and manipulated in terms of a world model or Ontology, and for which the DBA and GUI portions of the programming tasks can be specified and automatically generated from this Ontology thereby eliminating the triangle effect (and the need for the associated programming disciplines). Such an approach would make the resultant system robust and adaptive to change.